Registration marker with anti-rotation base for orthopedic surgical procedures

ABSTRACT

Bone tracking during orthopedic surgical procedures is described herein. An anti-rotation base of a registration marker of a mixed reality surgery system includes a surface configured to rest on and bear against a surface of a glenoid of a patient. The anti-rotation base also includes a plurality of pin receptacles that each define a respective pin axis. The pin axis of a first pin receptacle of the plurality of pin receptacles is not parallel to the pin axis of a second pin receptacle of the plurality of pin receptacles. Additionally, the anti-rotation base includes a mounting receptacle configured to receive the registration marker.

This application claims the benefit of U.S. Provisional PatentApplication No. 63/020,800, filed 6 May 2020, the entire contents ofwhich is incorporated herein by reference.

TECHNICAL FIELD

This disclosure relates to bone tracking during orthopedic surgicalprocedures.

BACKGROUND

Surgical joint repair procedures involve repair and/or replacement of adamaged or diseased joint. Many times, a surgical joint repairprocedure, such as joint arthroplasty as an example, involves replacingthe damaged joint with a prosthetic that is implanted into the patient'sbone. Proper selection of a prosthetic that is appropriately sized andshaped and proper positioning of that prosthetic to ensure an optimalsurgical outcome can be challenging. To assist with positioning, thesurgical procedure often involves the use of surgical instruments tocontrol the shaping of the surface of the damaged bone and cutting ordrilling of bone to accept the prosthetic.

Virtual visualization tools are available to surgeons that usethree-dimensional modeling of bone shapes to facilitate preoperativeplanning for joint repairs and replacements. These tools can assistsurgeons with the design and/or selection of surgical guides andimplants that closely match the patient's anatomy and can improvesurgical outcomes by customizing a surgical plan for each patient.

SUMMARY

This disclosure describes a variety of techniques for intra-operativesurgical guidance using mixed reality (MR)-based visualization forsurgical joint repair procedures. The techniques may be usedindependently or in various combinations to support particular phases orsettings for surgical joint repair procedures or provide a multi-facetedecosystem to support surgical joint repair procedures. As describedbelow, a registration marker is affixed to bone of a patient by ananti-rotation based. A three-dimensional geometric marker is mounted onanchor pin fixed in an anti-rotation base configured to rest on and bearagainst surface of the bone (e.g., a superior region of glenoid, etc.)to resist rotation. In some examples, the anti-rotation base is designedto conform to patient-specific anatomy to promote proper orientation. Insome examples, the anti-rotation base defines anchor holes are offsetrelative to each other so that the anchor rods do not interfere witheach other. The registration marker provides a reference for atransformation matrix used to align virtual objects (e.g., virtualimages of a glenoid, etc.) to their corresponding physical objects.

An anti-rotation base of a registration marker of a mixed realitysurgery system may include a surface configured to rest on and bearagainst a surface of a glenoid of a patient. The anti-rotation base mayalso include a plurality of pin receptacles that each define arespective pin axis. The pin axis of a first pin receptacle of theplurality of pin receptacles may not be parallel to the pin axis of asecond pin receptacle of the plurality of pin receptacles. Additionally,the anti-rotation base may include a mounting receptacle configured toreceive the registration marker.

The details of various examples of the disclosure are set forth in theaccompanying drawings and the description below. Various features,objects, and advantages will be apparent from the description, drawings,and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an orthopedic surgical system according toan example of this disclosure.

FIG. 2 is a block diagram of an orthopedic surgical system that includesa mixed reality (MR) system, according to an example of this disclosure.

FIG. 3 is a flowchart illustrating example phases of a surgicallifecycle.

FIG. 4 is a schematic representation of a visualization device for usein a mixed reality (MR) system, according to an example of thisdisclosure.

FIG. 5 is a block diagram illustrating example components of avisualization device for use in a mixed reality (MR) system, accordingto an example of this disclosure.

FIG. 6 is a flowchart illustrating example steps in the preoperativephase of the surgical lifecycle.

FIG. 7 is a flowchart illustrating example stages of a shoulder jointrepair surgery.

FIG. 8 illustrates an example technique for registering a 3-dimensionalvirtual bone model with an observed real bone structure of a patientduring joint repair surgery.

FIG. 9 illustrates an example registration marker with an anti-rotationbase.

FIG. 10A is a perspective view of an example anti-rotation base.

FIG. 10B is a front view of an example anti-rotation base.

FIG. 10C is a right view of an example anti-rotation base.

FIG. 10D is a top view of an example anti-rotation base.

FIG. 10E is a bottom view of an example anti-rotation base.

FIGS. 11A and 11B illustrate the registration marker of FIG. 9 restingon and bearing against a surface of a glenoid.

FIG. 12 is cross-section view of an example of an anti-rotation baseaffixed to a glenoid using pins.

DETAILED DESCRIPTION

The disclosure is directed to a registration marker to attach a fiducialmarker to a bone (e.g., a glenoid, etc.) in a patient to provide areference for a transformation matrix used in a mixed reality system.The apparatus includes an anti-rotation base, a mounting rod, and thefiducial marker. In some examples, the anti-rotation based defines anarch configured to rest on and bear against a surface of the bone (e.g.,a superior region of the glenoid, etc.). The anti-rotation base alsodefines a mount hole to slidably couple the anti-rotation base to themounting rod. The arch is configured to straddle a portion of the boneon which the apparatus will be mounted. The mounting hole is configuredto accommodate the mounting rod such that the mounting rod, wheninstalled, passes through the bridge. In some examples, the mounting rodterminates in an attachment end to anchor the mounting rod to the bone.Additionally, in some examples, the anti-rotation base defines at leasttwo anchor holes (sometimes referred to as “pin holes” or “pinreceptacles”). The anchor holes are configured to, when installed,accommodate anchor rods (sometimes referred to as “pins”) that fix theorientation of the bridge to prevent movement of the apparatus. Theanchor holes are offset relative to each other so that the anchor rods,when installed, do not interfere with each other. The example fiducialmarker is a polyhedron with a pattern recognizable by a computer visionsystem (e.g., a QR code, a bar code, an image, etc.) on one or morefaces to facilitate aligning an orientation of patient with theorientation of the image(s) to be overlaid via the mixed reality system.The fiducial marker is affixed to the end of the mounting rod such that,when installed, the fiducial marker is spaced away from the surface ofthe patient.

The registration maker is used with a mixed reality (MR) visualizationsystem to assist with creation, implementation, verification, and/ormodification of a surgical plan before and during a surgical procedure.Because MR, or in some instances VR, may be used to interact with thesurgical plan, this disclosure may also refer to the surgical plan as a“virtual” surgical plan. Visualization tools other than or in additionto mixed reality visualization systems may be used in accordance withtechniques of this disclosure. A surgical plan, e.g., as generated bythe BLUEPRINT™ system, available from Wright Medical Group, N.V., oranother surgical planning platform, may include information defining avariety of features of a surgical procedure, such as features ofparticular surgical procedure steps to be performed on a patient by asurgeon according to the surgical plan including, for example, bone ortissue preparation steps and/or steps for selection, modification and/orplacement of implant components. Such information may include, invarious examples, dimensions, shapes, angles, surface contours, and/ororientations of implant components to be selected or modified bysurgeons, dimensions, shapes, angles, surface contours and/ororientations to be defined in bone or tissue by the surgeon in bone ortissue preparation steps, and/or positions, axes, planes, angle and/orentry points defining placement of implant components by the surgeonrelative to patient bone or tissue. Information such as dimensions,shapes, angles, surface contours, and/or orientations of anatomicalfeatures of the patient may be derived from imaging (e.g., x-ray, CT,MRI, ultrasound or other images), direct observation, or othertechniques. In some examples, the virtual

In this disclosure, the term “mixed reality” (MR) refers to thepresentation of virtual objects such that a user sees images thatinclude both real, physical objects and virtual objects. Virtual objectsmay include text, 2-dimensional surfaces, 3-dimensional models, or otheruser-perceptible elements that are not actually present in the physical,real-world environment in which they are presented as coexisting. Inaddition, virtual objects described in various examples of thisdisclosure may include graphics, images, animations or videos, e.g.,presented as 3D virtual objects or 2D virtual objects. Virtual objectsmay also be referred to as virtual elements. Such elements may or maynot be analogs of real-world objects. In some examples, in mixedreality, a camera may capture images of the real world and modify theimages to present virtual objects in the context of the real world. Insuch examples, the modified images may be displayed on a screen, whichmay be head-mounted, handheld, or otherwise viewable by a user. In someexamples, in mixed reality, see-through (e.g., transparent) holographiclenses, which may be referred to as waveguides, may permit the user toview real-world objects, i.e., actual objects in a real-worldenvironment, such as real anatomy, through the holographic lenses andalso concurrently view virtual objects.

The Microsoft HOLOLENS™ headset, available from Microsoft Corporation ofRedmond, Wash., is an example of a MR device that includes see-throughholographic lenses, sometimes referred to as waveguides, that permit auser to view real-world objects through the lens and concurrently viewprojected 3D holographic objects. The Microsoft HOLOLENS™ headset, orsimilar waveguide-based visualization devices, are examples of an MRvisualization device that may be used in accordance with some examplesof this disclosure. Some holographic lenses may present holographicobjects with some degree of transparency through see-through holographiclenses so that the user views real-world objects and virtual,holographic objects. In some examples, some holographic lenses may, attimes, completely prevent the user from viewing real-world objects andinstead may allow the user to view entirely virtual environments. Theterm mixed reality may also encompass scenarios where one or more usersare able to perceive one or more virtual objects generated byholographic projection. In other words, “mixed reality” may encompassthe case where a holographic projector generates holograms of elementsthat appear to a user to be present in the user's actual physicalenvironment.

In some examples, in mixed reality, the positions of some or allpresented virtual objects are related to positions of physical objectsin the real world. For example, a virtual object may be tethered to atable in the real world, such that the user can see the virtual objectwhen the user looks in the direction of the table but does not see thevirtual object when the table is not in the user's field of view. Insome examples, in mixed reality, the positions of some or all presentedvirtual objects are unrelated to positions of physical objects in thereal world. For instance, a virtual item may always appear in the topright of the user's field of vision, regardless of where the user islooking.

Augmented reality (AR) is similar to MR in the presentation of bothreal-world and virtual elements, but AR generally refers topresentations that are mostly real, with a few virtual additions to“augment” the real-world presentation. For purposes of this disclosure,MR is considered to include AR. For example, in AR, parts of the user'sphysical environment that are in shadow can be selectively brightenedwithout brightening other areas of the user's physical environment. Thisexample is also an instance of MR in that the selectively-brightenedareas may be considered virtual objects superimposed on the parts of theuser's physical environment that are in shadow.

Furthermore, in this disclosure, the term “virtual reality” (VR) refersto an immersive artificial environment that a user experiences throughsensory stimuli (such as sights and sounds) provided by a computer.Thus, in virtual reality, the user may not see any physical objects asthey exist in the real world. Video games set in imaginary worlds are acommon example of VR. The term “VR” also encompasses scenarios where theuser is presented with a fully artificial environment in which somevirtual object's locations are based on the locations of correspondingphysical objects as they relate to the user. Walk-through VR attractionsare examples of this type of VR.

The term “extended reality” (XR) is a term that encompasses a spectrumof user experiences that includes virtual reality, mixed reality,augmented reality, and other user experiences that involve thepresentation of at least some perceptible elements as existing in theuser's environment that are not present in the user's real-worldenvironment. Thus, the term “extended reality” may be considered a genusfor MR and VR. XR visualizations may be presented in any of thetechniques for presenting mixed reality discussed elsewhere in thisdisclosure or presented using techniques for presenting VR, such as VRgoggles.

Visualization tools are available that utilize patient image data togenerate three-dimensional models of bone contours to facilitatepreoperative planning for joint repairs and replacements. These toolsallow surgeons to design and/or select surgical guides and implantcomponents that closely match the patient's anatomy. These tools canimprove surgical outcomes by customizing a surgical plan for eachpatient. An example of such a visualization tool for shoulder repairs isthe BLUEPRINT™ system available from Wright Medical Group, N.V. TheBLUEPRINT™ system provides the surgeon with two-dimensional planar viewsof the bone repair region as well as a three-dimensional virtual modelof the repair region. The surgeon can use the BLUEPRINT™ system toselect, design or modify appropriate implant components, determine howbest to position and orient the implant components and how to shape thesurface of the bone to receive the components, and design, select ormodify surgical guide tool(s) or instruments to carry out the surgicalplan. The information generated by the BLUEPRINT™ system is compiled ina preoperative surgical plan for the patient that is stored in adatabase at an appropriate location (e.g., on a server in a wide areanetwork, a local area network, or a global network) where it can beaccessed by the surgeon or other care provider, including before andduring the actual surgery.

FIG. 1 is a block diagram of an orthopedic surgical system 100 accordingto an example of this disclosure. Orthopedic surgical system 100includes a set of subsystems. In the example of FIG. 1, the subsystemsinclude a virtual planning system 102, a planning support system 104, amanufacturing and delivery system 106, an intraoperative guidance system108, a medical education system 110, a monitoring system 112, apredictive analytics system 114, and a communications network 116. Inother examples, orthopedic surgical system 100 may include more, fewer,or different subsystems. For example, orthopedic surgical system 100 mayomit medical education system 110, monitor system 112, predictiveanalytics system 114, and/or other subsystems. In some examples,orthopedic surgical system 100 may be used for surgical tracking, inwhich case orthopedic surgical system 100 may be referred to as asurgical tracking system. In other cases, orthopedic surgical system 100may be generally referred to as a medical device system.

Users of orthopedic surgical system 100 may use virtual planning system102 to plan orthopedic surgeries. Users of orthopedic surgical system100 may use planning support system 104 to review surgical plansgenerated using orthopedic surgical system 100. Manufacturing anddelivery system 106 may assist with the manufacture and delivery ofitems needed to perform orthopedic surgeries. Intraoperative guidancesystem 108 provides guidance to assist users of orthopedic surgicalsystem 100 in performing orthopedic surgeries. Medical education system110 may assist with the education of users, such as healthcareprofessionals, patients, and other types of individuals. Pre- andpostoperative monitoring system 112 may assist with monitoring patientsbefore and after the patients undergo surgery. Predictive analyticssystem 114 may assist healthcare professionals with various types ofpredictions. For example, predictive analytics system 114 may applyartificial intelligence techniques to determine a classification of acondition of an orthopedic joint, e.g., a diagnosis, determine whichtype of surgery to perform on a patient and/or which type of implant tobe used in the procedure, determine types of items that may be neededduring the surgery, and so on.

The subsystems of orthopedic surgical system 100 (i.e., virtual planningsystem 102, planning support system 104, manufacturing and deliverysystem 106, intraoperative guidance system 108, medical education system110, pre- and postoperative monitoring system 112, and predictiveanalytics system 114) may include various systems. The systems in thesubsystems of orthopedic surgical system 100 may include various typesof computing systems, computing devices, including server computers,personal computers, tablet computers, smartphones, display devices,Internet of Things (IoT) devices, visualization devices (e.g., mixedreality (MR) visualization devices, virtual reality (VR) visualizationdevices, holographic projectors, or other devices for presentingextended reality (XR) visualizations), surgical tools, and so on. Aholographic projector, in some examples, may project a hologram forgeneral viewing by multiple users or a single user without a headset,rather than viewing only by a user wearing a headset. For example,virtual planning system 102 may include a MR visualization device andone or more server devices, planning support system 104 may include oneor more personal computers and one or more server devices, and so on. Acomputing system is a set of one or more computing systems configured tooperate as a system. In some examples, one or more devices may be sharedbetween the two or more of the subsystems of orthopedic surgical system100. For instance, in the previous examples, virtual planning system 102and planning support system 104 may include the same server devices.

In the example of FIG. 1, the devices included in the subsystems oforthopedic surgical system 100 may communicate using communicationnetwork 116. Communication network 116 may include various types ofcommunication networks including one or more wide-area networks, such asthe Internet, local area networks, and so on. In some examples,communication network 116 may include wired and/or wirelesscommunication links.

Many variations of orthopedic surgical system 100 are possible inaccordance with techniques of this disclosure. Such variations mayinclude more or fewer subsystems than the version of orthopedic surgicalsystem 100 shown in FIG. 1. For example, FIG. 2 is a block diagram of anorthopedic surgical system 200 that includes one or more mixed reality(MR) systems, according to an example of this disclosure. Orthopedicsurgical system 200 may be used for creating, verifying, updating,modifying and/or implementing a surgical plan. In some examples, thesurgical plan can be created preoperatively, such as by using a virtualsurgical planning system (e.g., the BLUEPRINT™ system), and thenverified, modified, updated, and viewed intraoperatively, e.g., using MRvisualization of the surgical plan. In other examples, orthopedicsurgical system 200 can be used to create the surgical plan immediatelyprior to surgery or intraoperatively, as needed. In some examples,orthopedic surgical system 200 may be used for surgical tracking, inwhich case orthopedic surgical system 200 may be referred to as asurgical tracking system. In other cases, orthopedic surgical system 200may be generally referred to as a medical device system.

In the example of FIG. 2, orthopedic surgical system 200 includes apreoperative surgical planning system 202, a healthcare facility 204(e.g., a surgical center or hospital), a storage system 206 and anetwork 208 that allows a user at healthcare facility 204 to accessstored patient information, such as medical history, image datacorresponding to the damaged joint or bone and various parameterscorresponding to a surgical plan that has been created preoperatively(as examples). Preoperative surgical planning system 202 may beequivalent to virtual planning system 102 of FIG. 1 and, in someexamples, may generally correspond to a virtual planning system similaror identical to the BLUEPRINT™ system.

In the example of FIG. 2, healthcare facility 204 includes a mixedreality (MR) system 212. In some examples of this disclosure, MR system212 includes one or more processing device(s) (P) 210 to providefunctionalities that will be described in further detail below.Processing device(s) 210 may also be referred to as processor(s). Inaddition, one or more users of MR system 212 (e.g., a surgeon, nurse, orother care provider) can use processing device(s) (P) 210 to generate arequest for a particular surgical plan or other patient information thatis transmitted to storage system 206 via network 208. In response,storage system 206 returns the requested patient information to MRsystem 212. In some examples, the users can use other processingdevice(s) to request and receive information, such as one or moreprocessing devices that are part of MR system 212, but not part of anyvisualization device, or one or more processing devices that are part ofa visualization device (e.g., visualization device 213) of MR system212, or a combination of one or more processing devices that are part ofMR system 212, but not part of any visualization device, and one or moreprocessing devices that are part of a visualization device (e.g.,visualization device 213) that is part of MR system 212.

In some examples, multiple users can simultaneously use MR system 212.For example, MR system 212 can be used in a spectator mode in whichmultiple users each use their own visualization devices so that theusers can view the same information at the same time and from the samepoint of view. In some examples, MR system 212 may be used in a mode inwhich multiple users each use their own visualization devices so thatthe users can view the same information from different points of view.

In some examples, processing device(s) 210 can provide a user interfaceto display data and receive input from users at healthcare facility 204.Processing device(s) 210 may be configured to control visualizationdevice 213 to present a user interface. Furthermore, processingdevice(s) 210 may be configured to control visualization device 213 topresent virtual images, such as 3D virtual models, 2D images, and so on.Processing device(s) 210 can include a variety of different processingor computing devices, such as servers, desktop computers, laptopcomputers, tablets, mobile phones and other electronic computingdevices, or processors within such devices. In some examples, one ormore of processing device(s) 210 can be located remote from healthcarefacility 204. In some examples, processing device(s) 210 reside withinvisualization device 213. In some examples, at least one of processingdevice(s) 210 is external to visualization device 213. In some examples,one or more processing device(s) 210 reside within visualization device213 and one or more of processing device(s) 210 are external tovisualization device 213.

In the example of FIG. 2, MR system 212 also includes one or more memoryor storage device(s) (M) 215 for storing data and instructions ofsoftware that can be executed by processing device(s) 210. Theinstructions of software can correspond to the functionality of MRsystem 212 described herein. In some examples, the functionalities of avirtual surgical planning application, such as the BLUEPRINT™ system,can also be stored and executed by processing device(s) 210 inconjunction with memory storage device(s) (M) 215. For instance, memoryor storage system 215 may be configured to store data corresponding toat least a portion of a virtual surgical plan. In some examples, storagesystem 206 may be configured to store data corresponding to at least aportion of a virtual surgical plan. In some examples, memory or storagedevice(s) (M) 215 reside within visualization device 213. In someexamples, memory or storage device(s) (M) 215 are external tovisualization device 213. In some examples, memory or storage device(s)(M) 215 include a combination of one or more memory or storage deviceswithin visualization device 213 and one or more memory or storagedevices external to the visualization device.

Network 208 may be equivalent to network 116. Network 208 can includeone or more wide area networks, local area networks, and/or globalnetworks (e.g., the Internet) that connect preoperative surgicalplanning system 202 and MR system 212 to storage system 206. Storagesystem 206 can include one or more databases that can contain patientinformation, medical information, patient image data, and parametersthat define the surgical plans. For example, medical images of thepatient's diseased or damaged bone typically are generatedpreoperatively in preparation for an orthopedic surgical procedure. Themedical images can include images of the relevant bone(s) taken alongthe sagittal plane and the coronal plane of the patient's body. Themedical images can include X-ray images, magnetic resonance imaging(MRI) images, computerized tomography (CT) images, ultrasound images,and/or any other type of 2D or 3D image that provides information aboutthe relevant surgical area. Storage system 206 also can include dataidentifying the implant components selected for a particular patient(e.g., type, size, etc.), surgical guides selected for a particularpatient, and details of the surgical procedure, such as entry points,cutting planes, drilling axes, reaming depths, etc. Storage system 206can be a cloud-based storage system (as shown) or can be located athealthcare facility 204 or at the location of preoperative surgicalplanning system 202 or can be part of MR system 212 or visualizationdevice (VD) 213, as examples.

MR system 212 can be used by a surgeon before (e.g., preoperatively) orduring the surgical procedure (e.g., intraoperatively) to create,review, verify, update, modify and/or implement a surgical plan. In someexamples, MR system 212 may also be used after the surgical procedure(e.g., postoperatively) to review the results of the surgical procedure,assess whether revisions are required, or perform other postoperativetasks. To that end, MR system 212 may include a visualization device 213that may be worn by the surgeon and (as will be explained in furtherdetail below) is operable to display a variety of types of information,including a 3D virtual image of the patient's diseased, damaged, orpostsurgical joint and details of the surgical plan, such as a 3Dvirtual image of the prosthetic implant components selected for thesurgical plan, 3D virtual images of entry points for positioning theprosthetic components, alignment axes and cutting planes for aligningcutting or reaming tools to shape the bone surfaces, or drilling toolsto define one or more holes in the bone surfaces, in the surgicalprocedure to properly orient and position the prosthetic components,surgical guides and instruments and their placement on the damagedjoint, and any other information that may be useful to the surgeon toimplement the surgical plan. MR system 212 can generate images of thisinformation that are perceptible to the user of the visualization device213 before and/or during the surgical procedure.

In some examples, MR system 212 includes multiple visualization devices(e.g., multiple instances of visualization device 213) so that multipleusers can simultaneously see the same images and share the same 3Dscene. In some such examples, one of the visualization devices can bedesignated as the master device and the other visualization devices canbe designated as observers or spectators. Any observer device can bere-designated as the master device at any time, as may be desired by theusers of MR system 212.

In this way, FIG. 2 illustrates a surgical planning system that includesa preoperative surgical planning system 202 to generate a virtualsurgical plan customized to repair an anatomy of interest of aparticular patient. For example, the virtual surgical plan may include aplan for an orthopedic joint repair surgical procedure, such as one of astandard total shoulder arthroplasty or a reverse shoulder arthroplasty.In this example, details of the virtual surgical plan may includedetails relating to at least one of preparation of glenoid bone orpreparation of humeral bone. In some examples, the orthopedic jointrepair surgical procedure is one of a stemless standard total shoulderarthroplasty, a stemmed standard total shoulder arthroplasty, a stemlessreverse shoulder arthroplasty, a stemmed reverse shoulder arthroplasty,an augmented glenoid standard total shoulder arthroplasty, and anaugmented glenoid reverse shoulder arthroplasty.

The virtual surgical plan may include a 3D virtual model correspondingto the anatomy of interest of the particular patient, a 3D model of oneor more tools, and/or a 3D model of a prosthetic component matched tothe particular patient to repair the anatomy of interest or selected torepair the anatomy of interest. In some examples, the 3D model mayinclude a point cloud or mesh (e.g., polygonal mesh, wireframe, etc.)that represents a feature of the corresponding object. As one example, a3D model of a patient's bone may include a point cloud or mesh thatrepresents a wall of the bone. As another example, a 3D model of apatient's bone may include a first point cloud or mesh that representsan inner wall of the bone and a second point cloud or mesh thatrepresents an outer wall of the bone. As another example, a 3D model ofa prosthetic component (e.g., an implant) may include a point cloud ormesh that represents an outer surface of at least a portion of theprosthetic component (e.g., the portion that is inserted into the bone).As another example, a 3D model of an implant tool may include a pointcloud or mesh that represents an outer surface of at least a portion ofthe implant tool (e.g., the portion that is inserted into the bone).

Furthermore, in the example of FIG. 2, the surgical planning systemincludes a storage system 206 to store data corresponding to the virtualsurgical plan. The surgical planning system of FIG. 2 also includes MRsystem 212, which may comprise visualization device 213. In someexamples, visualization device 213 is wearable by a user. In someexamples, visualization device 213 is held by a user, or rests on asurface in a place accessible to the user. MR system 212 may beconfigured to present a user interface via visualization device 213. Theuser interface is visually perceptible to the user using visualizationdevice 213. For instance, in one example, a screen of visualizationdevice 213 may display real-world images and the user interface on ascreen. In some examples, visualization device 213 may project virtual,holographic images onto see-through holographic lenses and also permit auser to see real-world objects of a real-world environment through thelenses. In other words, visualization device 213 may comprise one ormore see-through holographic lenses and one or more display devices thatpresent imagery to the user via the holographic lenses to present theuser interface to the user.

In some examples, visualization device 213 is configured such that theuser can manipulate the user interface (which is visually perceptible tothe user when the user is wearing or otherwise using visualizationdevice 213) to request and view details of the virtual surgical plan forthe particular patient, including a 3D virtual model of the anatomy ofinterest (e.g., a 3D virtual bone of the anatomy of interest) and a 3Dmodel of the prosthetic component selected to repair an anatomy ofinterest. In some such examples, visualization device 213 is configuredsuch that the user can manipulate the user interface so that the usercan view the virtual surgical plan intraoperatively, including (at leastin some examples) the 3D virtual model of the anatomy of interest (e.g.,a 3D virtual bone of the anatomy of interest). In some examples, MRsystem 212 can be operated in an augmented surgery mode in which theuser can manipulate the user interface intraoperatively so that the usercan visually perceive details of the virtual surgical plan projected ina real environment, e.g., on a real anatomy of interest of theparticular patient. In this disclosure, the terms real and real worldmay be used in a similar manner. For example, MR system 212 may presentone or more virtual objects that provide guidance for preparation of abone surface and placement of a prosthetic implant on the bone surface.Visualization device 213 may present one or more virtual objects in amanner in which the virtual objects appear to be overlaid on an actual,real anatomical object of the patient, within a real-world environment,e.g., by displaying the virtual object(s) with actual, real-worldpatient anatomy viewed by the user through holographic lenses. Forexample, the virtual objects may be 3D virtual objects that appear toreside within the real-world environment with the actual, realanatomical object.

FIG. 3 is a flowchart illustrating example phases of a surgicallifecycle 300. In the example of FIG. 3, surgical lifecycle 300 beginswith a preoperative phase (302). During the preoperative phase, asurgical plan is developed. The preoperative phase is followed by amanufacturing and delivery phase (304). During the manufacturing anddelivery phase, patient-specific items, such as parts and equipment,needed for executing the surgical plan are manufactured and delivered toa surgical site. In some examples, it is unnecessary to manufacturepatient-specific items in order to execute the surgical plan. Anintraoperative phase follows the manufacturing and delivery phase (306).The surgical plan is executed during the intraoperative phase. In otherwords, one or more persons perform the surgery on the patient during theintraoperative phase. The intraoperative phase is followed by thepostoperative phase (308). The postoperative phase includes activitiesoccurring after the surgical plan is complete. For example, the patientmay be monitored during the postoperative phase for complications.

As described in this disclosure, orthopedic surgical system 100 (FIG. 1)may be used in one or more of preoperative phase 302, the manufacturingand delivery phase 304, the intraoperative phase 306, and thepostoperative phase 308. For example, virtual planning system 102 andplanning support system 104 may be used in preoperative phase 302.Manufacturing and delivery system 106 may be used in the manufacturingand delivery phase 304. Intraoperative guidance system 108 may be usedin intraoperative phase 306. Some of the systems of FIG. 1 may be usedin multiple phases of FIG. 3. For example, medical education system 110may be used in one or more of preoperative phase 302, intraoperativephase 306, and postoperative phase 308; pre- and postoperativemonitoring system 112 may be used in preoperative phase 302 andpostoperative phase 308. Predictive analytics system 114 may be used inpreoperative phase 302 and postoperative phase 308.

As mentioned above, one or more of the subsystems of orthopedic surgicalsystem 100 may include one or more mixed reality (MR) systems, such asMR system 212 (FIG. 2). Each MR system may include a visualizationdevice. For instance, in the example of FIG. 2, MR system 212 includesvisualization device 213. In some examples, in addition to including avisualization device, an MR system may include external computingresources that support the operations of the visualization device. Forinstance, the visualization device of an MR system may becommunicatively coupled to a computing device (e.g., a personalcomputer, backpack computer, smartphone, etc.) that provides theexternal computing resources. Alternatively, adequate computingresources may be provided on or within visualization device 213 toperform necessary functions of the visualization device.

FIG. 4 is a schematic representation of visualization device 213 for usein an MR system, such as MR system 212 of FIG. 2, according to anexample of this disclosure. As shown in the example of FIG. 4,visualization device 213 can include a variety of electronic componentsfound in a computing system, including one or more processor(s) 514(e.g., microprocessors or other types of processing units) and memory516 that may be mounted on or within a frame 518. Furthermore, in theexample of FIG. 4, visualization device 213 may include a transparentscreen 520 that is positioned at eye level when visualization device 213is worn by a user. In some examples, screen 520 can include one or moreliquid crystal displays (LCDs) or other types of display screens onwhich images are perceptible to a surgeon who is wearing or otherwiseusing visualization device 213 via screen 520. Other display examplesinclude organic light emitting diode (OLED) displays. In some examples,visualization device 213 can operate to project 3D images onto theuser's retinas using techniques known in the art.

In some examples, screen 520 may include see-through holographic lenses.sometimes referred to as waveguides, that permit a user to seereal-world objects through (e.g., beyond) the lenses and also seeholographic imagery projected into the lenses and onto the user'sretinas by displays, such as liquid crystal on silicon (LCoS) displaydevices, which are sometimes referred to as light engines or projectors,operating as an example of a holographic projection system 538 withinvisualization device 213. In other words, visualization device 213 mayinclude one or more see-through holographic lenses to present virtualimages to a user. Hence, in some examples, visualization device 213 canoperate to project 3D images onto the user's retinas via screen 520,e.g., formed by holographic lenses. In this manner, visualization device213 may be configured to present a 3D virtual image to a user within areal-world view observed through screen 520, e.g., such that the virtualimage appears to form part of the real-world environment. In someexamples, visualization device 213 may be a Microsoft HOLOLENS™ headset,available from Microsoft Corporation, of Redmond, Wash., USA, or asimilar device, such as, for example, a similar MR visualization devicethat includes waveguides. The HOLOLENS™ device can be used to present 3Dvirtual objects via holographic lenses, or waveguides, while permittinga user to view actual objects in a real-world scene, i.e., in areal-world environment, through the holographic lenses.

Although the example of FIG. 4 illustrates visualization device 213 as ahead-wearable device, visualization device 213 may have other forms andform factors. For instance, in some examples, visualization device 213may be a handheld smartphone or tablet.

Visualization device 213 can also generate a user interface (UI) 522that is visible to the user, e.g., as holographic imagery projected intosee-through holographic lenses as described above. For example, UI 522can include a variety of selectable widgets 524 that allow the user tointeract with a mixed reality (MR) system, such as MR system 212 of FIG.2. Imagery presented by visualization device 213 may include, forexample, one or more 3D virtual objects. Details of an example of UI 522are described elsewhere in this disclosure. Visualization device 213also can include a speaker or other sensory devices 526 that may bepositioned adjacent the user's ears. Sensory devices 526 can conveyaudible information or other perceptible information (e.g., vibrations)to assist the user of visualization device 213.

Visualization device 213 can also include a transceiver 528 to connectvisualization device 213 to a processing device 510 and/or to network208 and/or to a computing cloud, such as via a wired communicationprotocol or a wireless protocol, e.g., Wi-Fi, Bluetooth, etc.Visualization device 213 also includes a variety of sensors to collectsensor data, such as one or more optical camera(s) 530 (or other opticalsensors) and one or more depth camera(s) 532 (or other depth sensors),mounted to, on or within frame 518. In some examples, the opticalsensor(s) 530 are operable to scan the geometry of the physicalenvironment in which user of MR system 212 is located (e.g., anoperating room) and collect two-dimensional (2D) optical image data(either monochrome or color). Depth sensor(s) 532 are operable toprovide 3D image data, such as by employing time of flight, stereo orother known or future-developed techniques for determining depth andthereby generating image data in three dimensions. Other sensors caninclude motion sensors 533 (e.g., Inertial Mass Unit (IMU) sensors,accelerometers, etc.) to assist with tracking movement.

MR system 212 processes the sensor data so that geometric,environmental, textural, etc. landmarks (e.g., corners, edges or otherlines, walls, floors, objects) in the user's environment or “scene” canbe defined and movements within the scene can be detected. As anexample, the various types of sensor data can be combined or fused sothat the user of visualization device 213 can perceive 3D images thatcan be positioned, or fixed and/or moved within the scene. When fixed inthe scene, the user can walk around the 3D image, view the 3D image fromdifferent perspectives, and manipulate the 3D image within the sceneusing hand gestures, voice commands, gaze line (or direction) and/orother control inputs. As another example, the sensor data can beprocessed so that the user can position a 3D virtual object (e.g., abone model) on an observed physical object in the scene (e.g., asurface, the patient's real bone, etc.) and/or orient the 3D virtualobject with other virtual images displayed in the scene. As yet anotherexample, the sensor data can be processed so that the user can positionand fix a virtual representation of the surgical plan (or other widget,image or information) onto a surface, such as a wall of the operatingroom. Yet further, the sensor data can be used to recognize surgicalinstruments and the position and/or location of those instruments.

Visualization device 213 may include one or more processors 514 andmemory 516, e.g., within frame 518 of the visualization device. In someexamples, one or more external computing resources 536 process and storeinformation, such as sensor data, instead of or in addition to in-frameprocessor(s) 514 and memory 516. In this way, data processing andstorage may be performed by one or more processors 514 and memory 516within visualization device 213 and/or some of the processing andstorage requirements may be offloaded from visualization device 213.Hence, in some examples, one or more processors that control theoperation of visualization device 213 may be within the visualizationdevice, e.g., as processor(s) 514. Alternatively, in some examples, atleast one of the processors that controls the operation of visualizationdevice 213 may be external to the visualization device, e.g., asprocessor(s) 210. Likewise, operation of visualization device 213 may,in some examples, be controlled in part by a combination one or moreprocessors 514 within the visualization device and one or moreprocessors 210 external to the visualization device.

For instance, in some examples, when visualization device 213 is in thecontext of FIG. 2, processing of the sensor data can be performed byprocessing device(s) 210 in conjunction with memory or storage device(s)(M) 215. In some examples, processor(s) 514 and memory 516 mounted toframe 518 may provide sufficient computing resources to process thesensor data collected by cameras 530, 532 and motion sensors 533. Insome examples, the sensor data can be processed using a SimultaneousLocalization and Mapping (SLAM) algorithm, or other known orfuture-developed algorithm for processing and mapping 2D and 3D imagedata and tracking the position of visualization device 213 in the 3Dscene. In some examples, image tracking may be performed using sensorprocessing and tracking functionality provided by the MicrosoftHOLOLENS™ system, e.g., by one or more sensors and processors 514 withina visualization device 213 substantially conforming to the MicrosoftHOLOLENS™ device or a similar mixed reality (MR) visualization device.

In some examples, MR system 212 can also include user-operated controldevice(s) 534 that allow the user to operate MR system 212, use MRsystem 212 in spectator mode (either as master or observer), interactwith UI 522 and/or otherwise provide commands or requests to processingdevice(s) 210 or other systems connected to network 208. As examples,the control device(s) 234 can include a microphone, a touch pad, acontrol panel, a motion sensor or other types of control input deviceswith which the user can interact.

FIG. 5 is a block diagram illustrating example components ofvisualization device 213 for use in a MR system. In the example of FIG.5, visualization device 213 includes processors 514, a power supply 600,display device(s) 602, speakers 604, microphone(s) 606, input device(s)608, output device(s) 610, storage device(s) 612, sensor(s) 614, andcommunication devices 616. In the example of FIG. 5, sensor(s) 616 mayinclude depth sensor(s) 532, optical sensor(s) 530, motion sensor(s)533, and orientation sensor(s) 618. Optical sensor(s) 530 may includecameras, such as Red-Green-Blue (RGB) video cameras, infrared cameras,or other types of sensors that form images from light. Display device(s)602 may display imagery to present a user interface to the user.

Speakers 604, in some examples, may form part of sensory devices 526shown in FIG. 4. In some examples, display devices 602 may includescreen 520 shown in FIG. 4. For example, as discussed with reference toFIG. 4, display device(s) 602 may include see-through holographiclenses, in combination with projectors, that permit a user to seereal-world objects, in a real-world environment, through the lenses, andalso see virtual 3D holographic imagery projected into the lenses andonto the user's retinas, e.g., by a holographic projection system. Inthis example, virtual 3D holographic objects may appear to be placedwithin the real-world environment. In some examples, display devices 602include one or more display screens, such as LCD display screens, OLEDdisplay screens, and so on. The user interface may present virtualimages of details of the virtual surgical plan for a particular patient.

In some examples, a user may interact with and control visualizationdevice 213 in a variety of ways. For example, microphones 606, andassociated speech recognition processing circuitry or software, mayrecognize voice commands spoken by the user and, in response, performany of a variety of operations, such as selection, activation, ordeactivation of various functions associated with surgical planning,intra-operative guidance, or the like. As another example, one or morecameras or other optical sensors 530 of sensors 614 may detect andinterpret gestures to perform operations as described above. As afurther example, sensors 614 may sense gaze direction and performvarious operations as described elsewhere in this disclosure. In someexamples, input devices 608 may receive manual input from a user, e.g.,via a handheld controller including one or more buttons, a keypad, atouchscreen, joystick, trackball, and/or other manual input media, andperform, in response to the manual user input, various operations asdescribed above.

As discussed above, surgical lifecycle 300 may include a preoperativephase 302 (FIG. 3). One or more users may use orthopedic surgical system100 in preoperative phase 302. For instance, orthopedic surgical system100 may include virtual planning system 102 to help the one or moreusers generate a virtual surgical plan that may be customized to ananatomy of interest of a particular patient. As described herein, thevirtual surgical plan may include a 3-dimensional virtual model thatcorresponds to the anatomy of interest of the particular patient and a3-dimensional model of one or more prosthetic components matched to theparticular patient to repair the anatomy of interest or selected torepair the anatomy of interest. The virtual surgical plan also mayinclude a 3-dimensional virtual model of guidance information to guide asurgeon in performing the surgical procedure, e.g., in preparing bonesurfaces or tissue and placing implantable prosthetic hardware relativeto such bone surfaces or tissue.

FIG. 6 is a flowchart illustrating example steps in preoperative phase302 of surgical lifecycle 300. In other examples, preoperative phase 302may include more, fewer, or different steps. Moreover, in otherexamples, one or more of the steps of FIG. 6 may be performed indifferent orders. In some examples, one or more of the steps may beperformed automatically within a surgical planning system such asvirtual planning system 102 (FIG. 1) or 202 (FIG. 2).

In the example of FIG. 6, a model of the area of interest is generated(800). For example, a scan (e.g., a CT scan, MRI scan, or other type ofscan) of the area of interest may be performed. For example, if the areaof interest is the patient's shoulder, a scan of the patient's shouldermay be performed. The virtual planning system may generate a virtualmodel (e.g., a three-dimensional virtual model) of the area of interestbased on the scan. Furthermore, a pathology in the area of interest maybe classified (802). In some examples, the pathology of the area ofinterest may be classified based on the scan of the area of interest.For example, if the area of interest is the user's shoulder, a surgeonmay determine what is wrong with the patient's shoulder based on thescan of the patient's shoulder and provide a shoulder classificationindicating the diagnosis, e.g., such as primary glenoid humeralosteoarthritis (PGHOA), rotator cuff tear arthropathy (RCTA)instability, massive rotator cuff tear (MRCT), rheumatoid arthritis,post-traumatic arthritis, and osteoarthritis.

Additionally, a surgical plan may be selected based on the pathology(804). The surgical plan is a plan to address the pathology. Forinstance, in the example where the area of interest is the patient'sshoulder, the surgical plan may be selected from an anatomical shoulderarthroplasty, a reverse shoulder arthroplasty, a post-trauma shoulderarthroplasty, or a revision to a previous shoulder arthroplasty. Thesurgical plan may then be tailored to patient (806). For instance,tailoring the surgical plan may involve selecting and/or sizing surgicalitems needed to perform the selected surgical plan. Additionally, thesurgical plan may be tailored to the patient in order to address issuesspecific to the patient, such as the presence of osteophytes. Asdescribed in detail elsewhere in this disclosure, one or more users mayuse mixed reality systems of orthopedic surgical system 100 to tailorthe surgical plan to the patient.

The surgical plan may then be reviewed (808). For instance, a consultingsurgeon may review the surgical plan before the surgical plan isexecuted. As described in detail elsewhere in this disclosure, one ormore users may use mixed reality (MR) systems of orthopedic surgicalsystem 100 to review the surgical plan. In some examples, a surgeon maymodify the surgical plan using an MR system by interacting with a UI anddisplayed elements, e.g., to select a different procedure, change thesizing, shape or positioning of implants, or change the angle, depth oramount of cutting or reaming of the bone surface to accommodate animplant.

FIG. 7 describes an example surgical process for a shoulder surgery thatmay use a registration marker with an anti-rotation based to register avirtual image to a physical space. The surgeon may wear or otherwise usevisualization device 213 during each step of the surgical process. Inother examples, a shoulder surgery may include more, fewer, or differentsteps. For example, a shoulder surgery may include step for adding abone graft, adding cement, and/or other steps. In some examples,visualization device 213 may present virtual guidance to guide thesurgeon, nurse, or other users, through the steps in the surgicalworkflow.

In the example of FIG. 7, a surgeon performs an incision process (1900).During the incision process, the surgeon makes a series of incisions toexpose a patient's shoulder joint. In some examples, an MR system (e.g.,MR system 212, MR system 1800A, etc.) may help the surgeon perform theincision process, e.g., by displaying virtual guidance imageryillustrating how to where to make the incision.

Furthermore, in the example of FIG. 7, the surgeon may perform a humeruscut process (1902). During the humerus cut process, the surgeon mayremove a portion of the humeral head of the patient's humerus. Removingthe portion of the humeral head may allow the surgeon to access thepatient's glenoid. Additionally, removing the portion of the humeralhead may allow the surgeon to subsequently replace the portion of thehumeral head with a humeral implant compatible with a glenoid implantthat the surgeon plans to implant in the patient's glenoid.

As discussed above, the humerus preparation process may enable thesurgeon to access the patient's glenoid. In the example of FIG. 7, afterperforming the humerus preparation process, the surgeon may perform aregistration process that registers a virtual glenoid object with thepatient's actual glenoid bone (1904) in the field of view presented tothe surgeon by visualization device 213.

FIG. 8 illustrates an example of a technique 2500 for registering a 3Dvirtual bone model with a real observed bone structure of a patientusing physical markers (e.g., any combination of passive and activephysical markers). In other words, FIG. 8 is an example of a processflow, e.g., performed by visualization device 213, for registering avirtual bone model with an observed bone that is implemented in a mixedreality system, such as the mixed reality system 212 of FIG. 2. 3Dvirtual bone model may be a model of all or part of one or more bones.The process flow of FIG. 8 may be performed as part of the registrationprocess of step 1904 of FIG. 7.

In operation, the practitioner may place one or more physical markers atspecific positions (e.g., the registration marker 900 of FIGS. 9, 11A,and 11B below). In some examples, MR system 212 may output instructionsas to where the practitioner should place the physical markers. Forexample, a physical marker may be placed such that it rests on and bearsagainst a surface of a superior of the glenoid. The prescribed locationsmay correspond to specific locations on a virtual model that correspondsto the observed bone structure.

MR system 212 may utilize data from one or more sensors (e.g., one ormore of sensors 614 of visualization device 213 of FIG. 5) to identifythe location of the registration marker(s) (e.g., the registrationmarker 900 of FIGS. 9, 11A and 11B below) (2510). For instance, MRsystem 212 may use data generated by any combination of depth sensors532 and/or optical sensors 530 to identify a specific position (e.g.,coordinates) the registration marker(s). As one specific example, MRsystem 212 may utilize optical data generated by optical sensors 530 toidentify a centroid of the registration marker. MR system 212 may thenutilize depth data generated by depth sensors 532 and/or optical datagenerated by optical sensors 530 to determine a position and/ororientation of the identified centroid. MR system 212 may determine adistance between the centroid and an attachment point of theregistration marker. Based on the determined distance (i.e., between thecentroid and the attachment point) and the determinedposition/orientation of the centroid, MR system 212 may determine aposition/orientation of the attachment point.

MR system 212 may register the virtual model with the observed anatomybased on the identified positions (2512) of the registration marker(s).For instance, where the registration marker(s) is/are placed on theobserved bone structure at locations that correspond to specificlocation(s) on the virtual model that corresponds to the observed bonestructure, MR system 212 may generate a transformation matrix betweenthe virtual model and the observed bone structure 212. Thistransformation matrix may allow for translation along the x, y, and zaxes of the virtual model and rotation about the x, y and z axes inorder to achieve and maintain alignment between the virtual and observedbones. In some examples, after registration is complete, MR system 212utilize the results of the registration to perform simultaneouslocalization and mapping (SLAM) to maintain alignment of the virtualmodel to the corresponding observed object.

In some examples, the practitioner may remove the physical markers(e.g., after registration is complete). For instance, after MR system212 has completed the registration process using the physical markers,MR system 212 may output an indication that the physical markers may beremoved. In example where the physical markers are removed, MR system212 may subsequently maintain the registration of the virtual bone modelto the observed bone using virtual markers or any other suitabletracking technique.

In some examples, the practitioner may not remove the physical markersuntil a later point in the surgery. For instance, the practitioner maynot remove the physical markers until registration of the virtual modelto the observed bone is no longer required (e.g., after all virtualguidance that uses the registration has been displayed and correspondingsurgical steps have been completed).

In some examples, MR system 212 may be able to maintain the registrationbetween a virtual bone model and observed bone (e.g., glenoid, humerus,or other bone structure) throughout the procedure. However, in somecases, MR system 212 may lose, or otherwise be unable to maintain, theregistration between the virtual bone model and observed bone. Forinstance, MR system 212 may lose track of the registration marker(s).This loss may be the result of any number of factors including, but notlimited to, body fluids (e.g., blood) occluding the markers, the markersbecoming dislodged (e.g., a physical marker being knocked out ofposition), and the like. As such, MR system 212 may periodicallydetermine whether registration has been lost (2516).

In some examples, MR system 212 may determine that registration has beenlost where a confidence distance between a virtual point and acorresponding physical point exceeds a threshold confidence distance(e.g., a clinical value). MR system 212 may periodically determine theconfidence distance as a value that represents the accuracy of thecurrent registration. For instance, MR system 212 may determine that adistance between a virtual point and a corresponding physical point isless than 3 mm.

In some examples, MR system 212 may output a representation of theconfidence distance. As one example, MR system 212 may causevisualization device 213 to display a numerical value of the confidencedistance. As another example, MR system 212 may cause visualizationdevice 213 to display a graphical representation of the confidencedistance relative to the threshold confidence distance (e.g., display agreen circle if the confidence distance is less than half of thethreshold confidence distance, display a yellow circle if the confidencedistance is between half of the threshold confidence distance and thethreshold confidence distance, and display a red circle if theconfidence distance greater than the threshold confidence distance).

In some examples, MR system 212 may utilize the same thresholdconfidence distance throughout a surgical procedure. In some examples,MR system 212 may utilize different threshold confidence distances forvarious parts a surgical procedure. For instance, MR system 212 mayutilize a first threshold confidence distance for a first set of worksteps and use a second threshold confidence distance (that is differentthan the first threshold confidence distance) for a first set of worksteps for a second set of work steps.

Where registration has not been lost (“No” branch of 2516), MR system212 may continue to display virtual guidance (2514). However, where MRsystem 212 loses registration (“Yes” branch of 2516), MR system 212 mayperform one or more actions to re-register the virtual bone model to theobserved bone. As one example, MR system 212 may automatically attemptto perform the registration process without further action from thepractitioner. For instance, where physical markers have not beenremoved. MR system 212 may perform the registration process using thephysical markers. Alternatively, where the physical markers have beenremoved (or were never placed), MR system 212 may output a request forthe practitioner to place the physical markers. As such, MR system 212may be considered to periodically register the virtual model with theobserved bone.

In some examples, as opposed to automatically attempting re-registrationwhere registration is lost, MR system 212 may selectively performre-registration based on whether registration is still needed (2518). Insome examples, MR system 212 may determine that registration is stillneeded if additional virtual guidance will be displayed. Where MR system212 determines that registration is no longer needed (“No” branch of2518), MR system 212 may end the registration procedure.

As described above, MR system 212 may utilize any combination of virtualand physical markers to enable registration of virtual models tocorresponding observed structures. MR system 212 may use any of themarkers to perform an initial registration and, where needed, MR system212 may use any of the markers to perform a re-registration. The markersused for the initial registration may be the same as or may be differentthan the markers used for any re-registrations.

In some examples, to enhance the accuracy and quality of registration,during the initialization stage of the registration process, MR system212 can compute and display spatial constraints for user head pose andorientation. These constraints can be computed in real time and dependon the position of the user, and/or the orientation, and/or the distanceto the observed bone, and/or the depth camera characteristics. Forexample, MR system 212 may prompt the user to move closer to theobserved bone, to adjust the head position so that the user's gaze lineis perpendicular to the surface of interest of the observed bone, or tomake any other adjustments that can be useful to enhance theregistration process and which may depend on the particular surgicalapplication and/or the attributes of the particular anatomy of interestand/or the characteristics of the optical and depth sensors that areemployed in MR system 212.

In some examples, depth camera(s) 532 detect distance by using astructured light approach or time of flight of an optical signal havinga suitable wavelength. In general, the wavelength of the optical signalis selected so that penetration of the surface of the observed anatomyby the optical signal transmitted by depth camera(s) 532 is minimized.It should be understood, however, that other known or future developedtechniques for detecting distance also can be employed.

As discussed below, the registration techniques described herein may beperformed for any pair of virtual model and observed object. As oneexample, an MR system may utilize the registration techniques toregister a virtual model of a bone to an observed bone. As anotherexample, an MR system may utilize the registration techniques toregister a virtual model of an implant to an observed implant. An MRsystem may utilize the registration techniques to register a virtualmodel of a tool to an observed tool.

In some examples, an MR system may perform the registration techniquesonce for a particular pair of a virtual model and an observed object(e.g., within a particular surgical procedure). For instance, an MRsystem may register a virtual model of a glenoid with an observedglenoid and utilize the registration to provide virtual guidance formultiple steps of a surgical procedure. In some examples, an MR systemmay perform the registration techniques multiple times for a particularpair of a virtual model and an observed object (e.g., within aparticular surgical procedure). For instance, an MR system may firstregister a virtual model of a glenoid with an observed glenoid andutilize the registration to provide virtual guidance for one or moresteps of a surgical procedure. Then, for example, after material hasbeen removed from the glenoid (e.g., via reaming), the MR system mayregister another virtual model of the glenoid (that accounts for theremoved material) with an observed glenoid and use the subsequentregistration to provide virtual guidance for one or more other steps ofthe surgical procedure.

The registration process may be used in conjunction with the virtualplanning processes and/or intra-operative guidance described elsewherein this disclosure. Thus, in one example, a virtual surgical plan isgenerated or otherwise obtained to repair an anatomy of interest of aparticular patient (e.g., the shoulder joint of the particular patient).In instances where the virtual surgical plan is obtained, anothercomputing system may generate the virtual surgical plan and an MR system(e.g., MR system 212) or other computing system obtains the virtualsurgical plan from a computer readable medium, such as a communicationmedium or a non-transitory storage medium. In this example, the virtualsurgical plan may include a 3D virtual model of the anatomy of interestgenerated based on preoperative image data and a prosthetic componentselected for the particular patient to repair the anatomy of interest.Furthermore, in this example, a user may use a MR system (e.g., MRsystem 212) to implement the virtual surgical plan. In this example, aspart of using the MR system, the user may request the virtual surgicalplan for the particular patient.

Additionally, the user may view virtual images of the surgical planprojected within a real environment. For example, MR system 212 maypresent 3D virtual objects such that the objects appear to reside withina real environment, e.g., with real anatomy of a patient, as describedin various examples of this disclosure. In this example, the virtualimages of the surgical plan may include one or more of the 3D virtualmodel of the anatomy of interest, a 3D model of the prostheticcomponent, and virtual images of a surgical workflow to repair theanatomy of interest. Furthermore, in this example, the user may registerthe 3D virtual model with a real anatomy of interest of the particularpatient. The user may then implement the virtually generated surgicalplan to repair the real anatomy of interest based on the registration.In other words, in the augmented surgery mode, the user can use thevisualization device to align the 3D virtual model of the anatomy ofinterest with the real anatomy of interest.

In some examples, as part of registering the 3D virtual model with thereal anatomy of interest, the 3D virtual model can be aligned (e.g., bythe user) with the real anatomy of interest and generate atransformation matrix between the 3D virtual model and the real anatomyof interest based on the alignment. The transformation matrix provides acoordinate system for translating the virtually generated surgical planto the real anatomy of interest. For instance, the registration processmay allow the user to view steps of the virtual surgical plan projectedon the real anatomy of interest. For instance, the alignment of the 3Dvirtual model with the real anatomy of interest may generate atransformation matrix that may allow the user to view steps of thevirtual surgical plan (e.g., identification of an entry point forpositioning a prosthetic implant to repair the real anatomy of interest)projected on the real anatomy of interest.

With continued reference to FIG. 7, after performing the registrationprocess, the surgeon may perform a reaming axis drilling process (1906).During the reaming axis drilling process, the surgeon may drill areaming axis guide pin hole in the patient's glenoid to receive areaming guide pin. At a later stage of the shoulder surgery, the surgeonmay insert a reaming axis pin into the reaming axis guide pin hole. Insome examples, an MR system (e.g., MR system 212, MR system 1800A, etc.)may present a virtual reaming axis to help the surgeon perform thedrilling in alignment with the reaming axis and thereby place thereaming guide pin in the correct location and with the correctorientation.

The surgeon may perform the reaming axis drilling process in one ofvarious ways. For example, the surgeon may perform a guide-based processto drill the reaming axis pin hole. In the case, a physical guide isplaced on the glenoid to guide drilling of the reaming axis pin hole. Inother examples, the surgeon may perform a guide-free process, e.g., withpresentation of a virtual reaming axis that guides the surgeon to drillthe reaming axis pin hole with proper alignment. An MR system (e.g., MRsystem 212, MR system 1800A, etc.) may help the surgeon perform eitherof these processes to drill the reaming axis pin hole.

Furthermore, in the surgical process of FIG. 7, the surgeon may performa reaming axis pin insertion process (1908). During the reaming axis pininsertion process, the surgeon inserts a reaming axis pin into thereaming axis pin hole drilled into the patient's scapula. In someexamples, an MR system (e.g., MR system 212, MR system 1800A, etc.) maypresent virtual guidance information to help the surgeon perform thereaming axis pin insertion process.

After performing the reaming axis insertion process, the surgeon mayperform a glenoid reaming process (1910). During the glenoid reamingprocess, the surgeon reams the patient's glenoid. Reaming the patient'sglenoid may result in an appropriate surface for installation of aglenoid implant. In some examples, to ream the patient's glenoid, thesurgeon may affix a reaming bit to a surgical drill. The reaming bitdefines an axial cavity along an axis of rotation of the reaming bit.The axial cavity has an inner diameter corresponding to an outerdiameter of the reaming axis pin. After affixing the reaming bit to thesurgical drill, the surgeon may position the reaming bit so that thereaming axis pin is in the axial cavity of the reaming bit. Thus, duringthe glenoid reaming process, the reaming bit may spin around the reamingaxis pin. In this way, the reaming axis pin may prevent the reaming bitfrom wandering during the glenoid reaming process. In some examples,multiple tools may be used to ream the patient's glenoid. An MR system(e.g., MR system 212, MR system 1800A, etc.) may present virtualguidance to help the surgeon or other users to perform the glenoidreaming process. For example, the MR system may help a user, such as thesurgeon, select a reaming bit to use in the glenoid reaming process. Insome examples, the MR system present virtual guidance to help thesurgeon control the depth to which the surgeon reams the user's glenoid.In some examples, the glenoid reaming process includes a paleo reamingstep and a neo reaming step to ream different parts of the patient'sglenoid.

Additionally, in the surgical process of FIG. 7, the surgeon may performa glenoid implant installation process (1912). During the glenoidimplant installation process, the surgeon installs a glenoid implant inthe patient's glenoid. In some instances, when the surgeon is performingan anatomical shoulder arthroplasty, the glenoid implant has a concavesurface that acts as a replacement for the user's natural glenoid. Inother instances, when the surgeon is performing a reverse shoulderarthroplasty, the glenoid implant has a convex surface that acts as areplacement for the user's natural humeral head. In this reverseshoulder arthroplasty, the surgeon may install a humeral implant thathas a concave surface that slides over the convex surface of the glenoidimplant. As in the other steps of the shoulder surgery of FIG. 7, an MRsystem (e.g., MR system 212, MR system 1800A, etc.) may present virtualguidance to help the surgeon perform the glenoid installation process.

In some examples, the glenoid implantation process includes a process tofix the glenoid implant to the patient's scapula (1914). In someexamples, the process to fix the glenoid implant to the patient'sscapula includes drilling one or more anchor holes or one or more screwholes into the patient's scapula and positioning an anchor such as oneor more pegs or a keel of the implant in the anchor hole(s) and/orinserting screws through the glenoid implant and the screw holes,possibly with the use of cement or other adhesive. An MR system (e.g.,MR system 212, MR system 1800A, etc.) may present virtual guidance tohelp the surgeon with the process of fixing the glenoid implant theglenoid bone, e.g., including virtual guidance indicating anchor orscrew holes to be drilled or otherwise formed in the glenoid, and theplacement of anchors or screws in the holes.

Furthermore, in the example of FIG. 7, the surgeon may perform a humeruspreparation process (1916). During the humerus preparation process, thesurgeon prepares the humerus for the installation of a humerus implant.In instances where the surgeon is performing an anatomical shoulderarthroplasty, the humerus implant may have a convex surface that acts asa replacement for the patient's natural humeral head. The convex surfaceof the humerus implant slides within the concave surface of the glenoidimplant. In instances where the surgeon is performing a reverse shoulderarthroplasty, the humerus implant may have a concave surface and theglenoid implant has a corresponding convex surface. As describedelsewhere in this disclosure, an MR system (e.g., MR system 212, MRsystem 1800A, etc.) may present virtual guidance information to help thesurgeon perform the humerus preparation process.

Furthermore, in the example surgical process of FIG. 7, the surgeon mayperform a humerus implant installation process (1918). During thehumerus implant installation process, the surgeon installs a humerusimplant on the patient's humerus. As described elsewhere in thisdisclosure, an MR system (e.g., MR system 212, etc.) may present virtualguidance to help the surgeon perform the humerus preparation process.

After performing the humerus implant installation process, the surgeonmay perform an implant alignment process that aligns the installedglenoid implant and the installed humerus implant (1920). For example,in instances where the surgeon is performing an anatomical shoulderarthroplasty, the surgeon may nest the convex surface of the humerusimplant into the concave surface of the glenoid implant. In instanceswhere the surgeon is performing a reverse shoulder arthroplasty, thesurgeon may nest the convex surface of the glenoid implant into theconcave surface of the humerus implant. Subsequently, the surgeon mayperform a wound closure process (1922). During the wound closureprocess, the surgeon may reconnect tissues severed during the incisionprocess in order to close the wound in the patient's shoulder.

For a shoulder arthroplasty application, the registration process maystart by virtualization device 213 presenting the user with 3D virtualbone model of the patient's scapula and glenoid that was generated frompreoperative images of the patient's anatomy, e.g., by surgical planningsystem 102. The user can then manipulate 3D virtual bone model in amanner that aligns and orients 3D virtual bone model with the patient'sreal scapula and glenoid that the user is observing in the operatingenvironment. As such, in some examples, the MR system may receive userinput to aid in the initialization and/or registration. However,discussed above, in some examples, the MR system may perform theinitialization and/or registration process automatically (e.g., withoutreceiving user input to position the 3D bone model). For other types ofarthroplasty procedures, such as for the knee, hip, foot, ankle orelbow, different relevant bone structures can be displayed as virtual 3Dimages and aligned and oriented in a similar manner with the patient'sactual, real anatomy.

Regardless of the particular type of joint or anatomical structureinvolved, selection of the augment surgery mode initiates a procedurewhere 3D virtual bone model is registered with an observed bonestructure. In general, the registration procedure can be considered as aclassical optimization problem (e.g., either minimization ormaximization). For a shoulder arthroplasty procedure, known inputs tothe optimization (e.g., minimization) analysis are the 3D geometry ofthe observed patient's bone (derived from sensor data from thevisualization device 213, including depth data from the depth camera(s)532) and the geometry of the 3D virtual bone derived during the virtualsurgical planning state (such as by using the BLUEPRINT™ system). Otherinputs include details of the surgical plan (also derived during thevirtual surgical planning stage, such as by using the BLUEPRINT™system), such as the position and orientation of entry points, cuttingplanes, reaming axes and/or drilling axes, as well as reaming ordrilling depths for shaping the bone structure, the type, size and shapeof the prosthetic components, and the position and orientation at whichthe prosthetic components will be placed or, in the case of a fracture,the manner in which the bone structure will be rebuilt.

Upon selection of a particular patient from a welcome page of UIpresented by MR system 212 (FIG. 4), the surgical planning parametersassociated with that patient are connected with the patient's 3D virtualbone model, e.g., by one or more processors of visualization device 213.In the Augment Surgery mode, registration of 3D virtual bone model (withthe connected preplanning parameters) with the observed bone byvisualization device 213 allows the surgeon to visualize virtualrepresentations of the surgical planning parameters on the patient.

The optimization (e.g., minimization) analysis that is implemented toachieve registration of the 3D virtual bone model with the real bonegenerally is performed in two stages: an initialization stage and anoptimization (e.g., minimization) stage. During the initializationstage, the user approximately aligns the 3D virtual bone model with thepatient's real bone, such as by using gaze direction, hand gesturesand/or voice commands to position and orient, or otherwise adjust, thealignment of the virtual bone with the observed real bone. Theinitialization stage will be described in further detail below. Duringthe optimization (e.g., minimization) stage, which also will bedescribed in detail below, an optimization (e.g., minimization)algorithm is executed that uses information from the optical camera(s)530 and/or depth camera(s) 532 and/or any other acquisition sensor(e.g., motion sensors 533) to further improve the alignment of the 3Dmodel with the observed anatomy of interest. In some examples, theoptimization (e.g., minimization) algorithm can be a minimizationalgorithm, including any known or future-developed minimizationalgorithm, such as an Iterative Closest Point algorithm or a geneticalgorithm as examples.

In this way, in one example, a mixed reality surgical planning methodincludes generating a virtual surgical plan to repair an anatomy ofinterest of a particular patient. The virtual surgical plan including a3D virtual model of the anatomy of interest is generated based onpreoperative image data and a prosthetic component selected for theparticular patient to repair the anatomy of interest. Furthermore, inthis example, the method includes using a MR visualization system toimplement the virtual surgical plan. In this example, using the MRsystem may comprise requesting the virtual surgical plan for theparticular patient. Using the MR system also comprises viewing virtualimages of the surgical plan projected within a real environment. Forexample, visualization device 213 may be configured to present one ormore 3D virtual images of details of the surgical plan that areprojected within a real environment, e.g., such that the virtualimage(s) appear to form part of the real environment. The virtual imagesof the surgical plan may include the 3D virtual model of the anatomy ofinterest, a 3D model of the prosthetic component, and virtual images ofa surgical workflow to repair the anatomy of interest. Using the MRsystem may also include registering the 3D virtual model with a realanatomy of interest of the particular patient. Additionally, in thisexample, using the MR system may include implementing the virtuallygenerated surgical plan to repair the real anatomy of interest based onthe registration.

Furthermore, in some examples, the method comprises registering the 3Dvirtual model with the real anatomy of interest without using virtual orphysical markers. The method may also comprise using the registration totrack movement of the real anatomy of interest during implementation ofthe virtual surgical plan on the real anatomy of interest. The movementof the real anatomy of interest may be tracked without the use oftracking markers. In some instances, registering the 3D virtual modelwith the real anatomy of interest may comprise aligning the 3D virtualmodel with the real anatomy of interest and generating a transformationmatrix between the 3D virtual model and the real anatomy of interestbased on the alignment. The transformation matrix provides a coordinatesystem for translating the virtually generated surgical plan to the realanatomy of interest. In some examples, aligning may comprise virtuallypositioning a point of interest on a surface of the 3D virtual modelwithin a corresponding region of interest on a surface of the realanatomy of interest; and adjusting an orientation of the 3D virtualmodel so that a virtual surface shape associated with the point ofinterest is aligned with a real surface shape associated with thecorresponding region of interest. In some examples, aligning may furthercomprise rotating the 3D virtual model about a gaze line of the user.The region of interest may be an anatomical landmark of the anatomy ofinterest. The anatomy of interest may be a shoulder joint. In someexamples, the anatomical landmark is a center region of a glenoid.

In some examples, after a registration process is complete, a trackingprocess can be initiated that continuously and automatically verifiesthe registration between 3D virtual bone model and observed bonestructure during the Augment Surgery mode. During a surgery, many eventscan occur (e.g., patient movement, instrument movement, loss oftracking, etc.) that may disturb the registration between the 3Danatomical model and the corresponding observed patient anatomy or thatmay impede the ability of MR system 212 to maintain registration betweenthe model and the observed anatomy. Therefore, by implementing atracking feature, MR system 212 can continuously or periodically verifythe registration and adjust the registration parameters as needed. If MRsystem 212 detects an inappropriate registration (such as patientmovement that exceeds a threshold amount), the user may be asked tore-initiate the registration process.

As discussed above, MR system 212 initially registers the 3D virtualbone model with the observed bone structure using a registration marker.MR system 212 may continue to track a position of the registrationmarker and utilize said tracked position to guide surgical steps. Forinstance, MR system 212 may output, for display, a virtual axis at aposition relative to the position of the registration marker. As such,if the registration marker is rotated or otherwise moved during theprocedure, the surgical steps guided by MR system 212 based on theposition of the registration marker may be performed with reducedaccuracy. For example, if the registration marker is jostled or bumpedinto, causing it to rotate, during the procedure, MR system 212 maydisplay the virtual axis at an incorrect location on the bone, eventhough the virtual axis is still displayed at the correct relativeposition to the registration marker. As such, maintaining anon-rotating, fixed reference facilitates providing a stable alignmentwith minimal interruption to the procedure may be desirable.

In accordance with one or more aspects of this disclosure, aregistration marker may be attached to bone via an anti-rotation base.For instance, as discussed in further detail below, the anti-rotationbase may include a plurality of pin receptacles that each define arespective pin axis. At least two of the pin axes may not be parallel.As such, when pins are installed via the pin receptacles, theanti-rotation base will be secured to the bone via at least twonon-parallel pins. By virtue of having non-parallel pins, theanti-rotation base may better resist rotation (e.g., as compared to pinsthat are parallel). In this way, the registration marker may be moresecurely attached to bone, which may improve the accuracy of surgicalsteps guided based on a position of the registration marker.

FIG. 9 illustrates an example registration marker 900. In theillustrated example, registration marker 900 include a fiducial marker902, a mounting rod 904, and an anti-rotation based 906. Registrationmarker 900, when installed, may rest on and bear against surface of thebone (e.g., a superior region of glenoid, etc.) with fiducial marker 902is spaced away from the surface of the patient to provide a reference toalign a virtual object with its corresponding physical object (e.g., toalign a virtual representation of a glenoid in an MR system to thephysical location of the patient's glenoid, etc.).

Fiducial marker 902 includes indicia 908 one or more sides to facilitatelocating a centroid of registration marker 900 and/or an orientation ofregistration marker 900 in space. In the illustrated example, fiducialmarker 902 is a polyhedron with indicia 908 on multiple sides (e.g.,three sides, etc.) to facilitate an MR system establishing relativethree-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis).Indicia 908 are markings that are readable by a computer vision systemthat are, in some examples, decodable (e.g., QR codes, bar codes,images, etc.) to facilitate determining the orientation and location offiducial marker 902. In some examples, one or more of indicia 908 mayinclude markings that indicate aspects about their location on fiducialmarker 902. As one example, an indicia of indicia 908 may encode anindication that it is on a front face of fiducial marker 902. As anotherexample, an indicia of indicia 908 may encode an indication that it ison a top face of fiducial marker 902.

Mounting rod 904 includes a proximal end 910 and a distal end 912.Fiducial marker 902 is affixed or to integrated into the distal end 912.The proximal end 910 is configured to be slidably mountable toanti-rotation abase 906. In the illustrate example, mounting rod 904includes a flange portion 914 configured to limit a range in whichanti-rotation base 906 slides about mounting rod 904. In some example,the distal end 912 is configured to be anchored to a bone of thepatient.

As described below in FIGS. 10A-10E, anti-rotation based 906 isconfigured to straddle a bone of the patient to resist rotation ofregistration marker 900. The anti-rotation based 906 is defined by a topsurface 1002, side surfaces 1004 and 1006, a front surface 1008, a backsurface 1010, and a bottom surface 1012. In the illustrated example,front surface 1008 and back surface 1010 are parallel. In some examples,front surface 1008 and back surface 1010 are flat.

In the illustrated example, bottom surface 1012 is curved to define acavity 1014. When installed in a patient, at least a portion of a boneof the patient is within cavity 1014. In some examples, bottom surface1012 rests on and bear against a surface a glenoid of the patient suchthat a portion of the glenoid is within cavity 1014. For example, wheninstalled, cavity 1014 may straddle a superior region of the glenoid. Insome examples, cavity 1014 is parabolic. In some examples, a shape ofcavity 1014 is tailored to a shape of a bone of a particular patientthat anti-rotation based 906 is to straddle based on, for example,images and/or models of the bone. In some examples, the shape of cavity1014 may have predefine variations, where the particular shape of cavity1014 of a particular patient is selected based on pre-operative imagesof the patient. In the illustrated examples, the curve of bottom surface1004 has an apex 1016 that defines a narrow portion of anti-rotationbased 906. Apex 1016 also defines two sides 1020A and 1020B of bottomsurface 1012. In some examples, when anti-rotation base 906 is tailoredto a shape of the bone of a particular patient, sides 1020A and 1020Bmay be modified. In some examples, anti-rotation base 906 may bepre-manufactured in different sizes and/or shapes. In some suchexamples, different ones of anti-rotation bases 906 may be manufacturedsuch that sides 1020A and 1020B have different sizes and shapes. Forexample, a slope and/or curvature of one or more of sides 1020A and1020B may be changed and/or a length of one or more of sides 1020A and1020B may be elongated or shortened. While in the illustrated example,cavity 1014 is an arch, cavity 1014 may be any shape that facilitatesanti-rotation base 906 straddling a bone of a patient. In theillustrated examples, side 1020A of bottom surface 112 meets sidesurface 1004 at an apex 1022A and side 1020B of bottom surface 112 meetsside surface 1006 at an apex 1022B.

Anti-rotation base 906 defines a plurality of pin receptacles 1024A and1024B (collectively “pin receptacles 1024”) around respective pin axes1026A and 1026B. In the illustrated example, a first pin receptacle1024A extends from side surface 1004 through anti-rotation base 906 anda second pin receptacle 1022A extends from side surface 1006 throughanti-rotation base 906. Pin axes 1026A and 1026B are not parallel andare configured that when anti-rotation based 906 is installed in thepatient, pins (e.g., the pins 1102 of FIGS. 11 and 12 below) that areembedded into the bone do not interfere with each other (e.g., the pinaxes do not intersect). For example, one pin receptacle 1024A may beoffset from the other pin receptacle 1024B. In the illustrated example,anti-rotation base 906 defines two pin receptacles 1024A and 1024B.However, anti-rotation base 906 may have more or fewer pin receptacles.For instance, anti-rotation base 906 may include three, four, or morepin receptacles.

In the illustrated examples, anti-rotation base 906 defines a mountingreceptacle 1028 around a mounting axis 1030. Mounting axis 1030 is notparallel to pin axes 1026A and 1026B. In the illustrated example,mounting axis 1030 is an axis of symmetry of anti-rotation base 906relative the curve of bottom surface 1012. Mounting receptacle 1024extends from top surface 1002 to bottom surface 1012 thoughanti-rotation base 906. In some examples, mounting receptacle 1024extend from top surface 1002 to apex 1016 of the curve of bottom surface1012. Mounting receptacle 1028 is configured to slidably coupled tomounting pin 904 so that, when attached to the patient, mounting pin 904extends into cavity 1014 though anti-rotation base 906.

FIGS. 11A and 11B illustrate registration marker 900 of FIG. 9 restingon and bearing against a surface 1100 of a glenoid 1102. In theillustrated example, the anti-rotation base 906 is affixed to glenoid1102 by one or more pins 1104 configured to screw into glenoid 1102through pin receptacles 1024 such that glenoid 1102 is within cavity1014. Pins 1104 fix the orientation of anti-rotation base 906 to preventmovement of registration marker 900. FIG. 12 is cross-section view ofanti-rotation base 906 affixed to glenoid 1102 using pins 1104. Asillustrated in FIG. 12, when installed, pins 1104 do not interfere witheach other. In some examples, when pins 1004 are installed, tips of pins1004 contacts a side (e.g., side 1020A or side 1020B) of anti-rotationbases 906 opposite the corresponding pin receptacle (e.g., pinreceptacles 1024A and 1024B). For example, when installing pin 1004 viapin receptacle 1024A, a surgeon may know pin 1004 is deep enough whenthe tip of pin 1004 contacts sides 1020B. In such examples, sides 1020Aand 1020B may act as depth guides to facilitate attaching anti-rotationbase 906 without an explicit pin depth measurement. As illustrated inFIGS. 11A and 11B, mounting rod 904 screws into glenoid 1102 throughmounting receptacle 1028. Fiducial marker extends away from glenoid1102.

While the techniques been disclosed with respect to a limited number ofexamples, those skilled in the art, having the benefit of thisdisclosure, will appreciate numerous modifications and variations therefrom. For instance, it is contemplated that any reasonable combinationof the described examples may be performed. It is intended that theappended claims cover such modifications and variations as fall withinthe true spirit and scope of the invention.

It is to be recognized that depending on the example, certain acts orevents of any of the techniques described herein can be performed in adifferent sequence, may be added, merged, or left out altogether (e.g.,not all described acts or events are necessary for the practice of thetechniques). Moreover, in certain examples, acts or events may beperformed concurrently, e.g., through multi-threaded processing,interrupt processing, or multiple processors, rather than sequentially.

In one or more examples, the functions described may be implemented inhardware, software, firmware, or any combination thereof. If implementedin software, the functions may be stored on or transmitted over as oneor more instructions or code on a computer-readable medium and executedby a hardware-based processing unit. Computer-readable media may includecomputer-readable storage media, which corresponds to a tangible mediumsuch as data storage media, or communication media including any mediumthat facilitates transfer of a computer program from one place toanother, e.g., according to a communication protocol. In this manner,computer-readable media generally may correspond to (1) tangiblecomputer-readable storage media which is non-transitory or (2) acommunication medium such as a signal or carrier wave. Data storagemedia may be any available media that can be accessed by one or morecomputers or one or more processors to retrieve instructions, codeand/or data structures for implementation of the techniques described inthis disclosure. A computer program product may include acomputer-readable medium.

By way of example, and not limitation, such computer-readable storagemedia can comprise RAM, ROM, EEPROM, CD-ROM or other optical diskstorage, magnetic disk storage, or other magnetic storage devices, flashmemory, or any other medium that can be used to store desired programcode in the form of instructions or data structures and that can beaccessed by a computer. Also, any connection is properly termed acomputer-readable medium. For example, if instructions are transmittedfrom a website, server, or other remote source using a coaxial cable,fiber optic cable, twisted pair, digital subscriber line (DSL), orwireless technologies such as infrared, radio, and microwave, then thecoaxial cable, fiber optic cable, twisted pair, DSL, or wirelesstechnologies such as infrared, radio, and microwave are included in thedefinition of medium. It should be understood, however, thatcomputer-readable storage media and data storage media do not includeconnections, carrier waves, signals, or other transitory media, but areinstead directed to non-transitory, tangible storage media. Disk anddisc, as used herein, includes compact disc (CD), laser disc, opticaldisc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wheredisks usually reproduce data magnetically, while discs reproduce dataoptically with lasers. Combinations of the above should also be includedwithin the scope of computer-readable media.

Operations described in this disclosure may be performed by one or moreprocessors, which may be implemented as fixed-function processingcircuits, programmable circuits, or combinations thereof, such as one ormore digital signal processors (DSPs), general purpose microprocessors,application specific integrated circuits (ASICs), field programmablegate arrays (FPGAs), or other equivalent integrated or discrete logiccircuitry. Fixed-function circuits refer to circuits that provideparticular functionality and are preset on the operations that can beperformed. Programmable circuits refer to circuits that can programmedto perform various tasks and provide flexible functionality in theoperations that can be performed. For instance, programmable circuitsmay execute instructions specified by software or firmware that causethe programmable circuits to operate in the manner defined byinstructions of the software or firmware. Fixed-function circuits mayexecute software instructions (e.g., to receive parameters or outputparameters), but the types of operations that the fixed-functioncircuits perform are generally immutable. Accordingly, the terms“processor” and “processing circuitry,” as used herein may refer to anyof the foregoing structures or any other structure suitable forimplementation of the techniques described herein.

Various examples have been described. These and other examples arewithin the scope of the following claims.

1: An anti-rotation base of a registration marker of a mixed realitysurgery system, the anti-rotation base comprising: a surface configuredto rest on and bear against a surface of a glenoid of a patient; aplurality of pin receptacles that each define a respective pin axis,wherein the pin axis of a first pin receptacle of the plurality of pinreceptacles is not parallel to the pin axis of a second pin receptacleof the plurality of pin receptacles; and a mounting receptacleconfigured to receive the registration marker. 2: The anti-rotation baseof claim 1, wherein the surface defines a cavity configured such that,when attached to the patient, a portion of the glenoid is within thecavity. 3: The anti-rotation base of claim 2, wherein the cavity isspecifically shaped for the patient. 4: The anti-rotation base of claim2, wherein the cavity has a parabolic shape. 5: The anti-rotation baseof claim 1, wherein the surface defines a cavity, and wherein themounting receptacle is configured to slidably coupled to a mounting pinof the registration marker so that, when attached to the patient, themounting pin extends into the cavity though the anti-rotation base. 6:The anti-rotation base of claim 1, wherein the anti-rotation basedefines a cavity and wherein a mounting axis of the mounting receptacledefines define an axis of symmetry of the cavity. 7: The anti-rotationbase of claim 1, wherein the surface defines a cavity, and wherein thefirst pin receptacle is positioned on a first side of the cavity and thesecond pin receptacle is positioned on a second side of the cavity. 8:The anti-rotation base of claim 1, wherein the mounting receptacledefines a mounting axis, and wherein, the pin axis of the first pinreceptacle and the pin axis of the second pin receptacle are notparallel to the mounting axis. 9: The anti-rotation base of claim 1,wherein the pin axis of the first pin receptacle and the pin axis of thesecond pin receptacle are defined such that, when installed, a firstanchor rod in the first pin receptacle does not interfere with a secondanchor rod in the second pin receptacle. 10: The anti-rotation base ofclaim 1, wherein the surface is configured to rest on and bear againstsurface of superior region of the glenoid of the patient. 11: Ananti-rotation base of a registration marker of a mixed reality surgerysystem, the anti-rotation base comprising: a bottom surface having acurve that defines a first bottom side and a second bottom side, whereinwhen the anti-rotation base is attached to a patient, the bottom surfacestraddles a glenoid of a patient; a first side surface; a second sidesurface parallel with the first side surface; a first end surface, afirst pin receptacle extending from the first end surface to the firstbottom side of the bottom surface; and a second end surface opposite thefirst end surface, a second pin receptacle extending from the second endsurface to the second bottom side of the bottom surface. 12: Theanti-rotation base of claim 11, further comprising a top surface, amounting receptacle extending from the top surface to the bottomsurface. 13: The anti-rotation base of claim 12, wherein the mountingreceptacle extends from the top surface to an apex of the curve of thebottom surface. 14: The anti-rotation base of claim 11, wherein thefirst and second pin receptacles each define a respective pin axis,wherein the pin axis of a first pin receptacle is not parallel to thepin axis of a second pin receptacle. 15: The anti-rotation base of claim14, further comprising a top surface, a mounting receptacle extendingfrom the top surface to the bottom surface, the mounting receptacledefining a mounting axis, wherein the pin axis of the first pinreceptacle and the pin axis of the second pin receptacle are notparallel to the mounting axis 16: The anti-rotation base of claim 14,wherein the pin axis of the first pin receptacle and the pin axis of thesecond pin receptacle are defined such that, when installed, a firstanchor rod in the first pin receptacle does not interfere with a secondanchor rod in the second pin receptacle. 17: The anti-rotation base ofclaim 16, wherein the pin axis of the first pin receptacle intersectsthe second bottom side of the bottom surface such that, when installed,a tip of the first anchor rod in the first pin receptacle contacts thesecond bottom side of the bottom surface, and wherein the pin axis ofthe second pin receptacle intersects the first bottom side of the bottomsurface such that, when installed, a tip of the second anchor rod in thesecond pin receptacle contacts the first bottom side of the bottomsurface. 18: The anti-rotation base of claim 11, wherein first andsecond bottom sides define a cavity configured such that, when theanti-rotation base is attached to the patient, a portion of the glenoidis within the cavity. 19: The anti-rotation base of claim 18, whereinthe cavity is specifically shaped for the patient. 20: The anti-rotationbase of claim 18, wherein the cavity has a parabolic shape. 21: Theanti-rotation base of claim 18, further comprising a top surface, amounting receptacle extending from the top surface to the bottomsurface, wherein a mounting axis of the mounting receptacle definesdefine an axis of symmetry of the cavity.